Speech and face-to-face communication - An introduction

This special issue was launched in parallel with the “speech and face to face communication” workshop organized in Grenoble in October 2008 and dedicated to the memory of Christian Benoît who died in 1998. The aim of this workshop was to show how and why speech communication must be increasingly studied in a face-to-face perspective in continuation of Christian Benoît’s researches. Speech communication is interactive. Partners involved in a spoken conversation indeed build a complex communicative act together which involves linguistic, emotional, expressive, and more generally cognitive and social dimensions. In this sense, it appears crucial to study spoken communication from an interactive point of view and the body of research on speech in interaction has recently grown actively. This issue focuses on face-to-face speech communication. Research works have demonstrated that this communicative situation is essential to language acquisition and development (e.g. naming). Faceto-face communication is in fact much more than speaking and speech is greatly influenced both in substance and content by this essential form of communication. Face-to-face communication is multimodal: interacting involves multimodality and nonverbal communication to a large extent. Speakers not only hear but also see each other producing sounds as well as facial and more generally body gestures. Gaze together with speech contribute to maintain mutual attention and to regulate turn-taking for example. Moreover, speech communication involves not only linguistic but also psychological, affective and social aspects of interaction. Face-to-face communication is situated: the true challenge of spoken communication is to take into account and integrate information not only from the speakers but also from the entire physical environment in which the interaction takes place. The communicative setting, the “task” in which the interlocutors are involved, their respective roles and the environmental conditions of the conversation indeed greatly influence how the spoken interaction unfolds. The present issue aims at synthesizing the most recent developments in this topic considering its various aspects from complementary perspectives: cognitive and neurocognitive (multisensory and perceptuo-motor interactions), linguistic (dialogic face to face interactions), paralinguistic (emotions and affects, turn-taking, mutual attention), computational (animated conversational agents, multimodal interacting communication systems). This issue focuses on a topic which has, to our knowledge, seldom been addressed in this form previously. Many related themes have been tackled, including e.g. audiovisual speech processing, dialog, multimodal communication systems, conversational agents, social robotics, and expressive communication. In this issue, we intended to focus on the convergence of these themes in a face-to-face interaction context in the speech communication area. Moreover, such an issue is timely, considering the very strong development of knowledge, techniques and systems in all the relevant areas mentioned above. We would like to dedicate this issue to the memory of Christian Benoît who’s scientific achievements in the field are recalled below. The present issue is divided into four main sections. The first section groups papers on avatars and augmented reality (Weiss et al.; Badin et al.; Heracleous et al.). Its aim is to present studies showing the potential benefits of augmented communication i.e. virtual interlocutors that supplement speech with multimodal behaviour. The second section deals with multimodal speech perception (Troille et al.; Fort et al.; Sato et al.) and presents studies investigating human-human speech communication is grounded in multimodality. The third section tackles the topic of manual gestures and prosody in communication (Colletta et al.; Cvejic et al.; Flecha-Garcia). Finally, the last section of the present issue presents studies analyzing the importance of mutual adaptation and attuning in interaction (Aubanel et al.; Kopp; Bailly et al.).